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; ABSTRACT 

i \ 

A model  is  presented  of  a Markov  process  whose  state  is  unknown  except 
• when  an  inspection  is  performed.  The  evolution  of  the  process  is 
governed  by  fixed  transition  probability  matrices  P and  Q under 
[ non-inspection  and  inspection,  respectively.  The  costs  of  i''spection 

and  non-inspection  depend  on  the  current  state.  The  objective  is  to 
characterize  inspection  policies  which  minimize  expected  tOi.ai  dis- 
I ’ counted  cost. 

* The  following  specific  models  are  presented.  Model  I is  a process 

which  starts  in  state  0 and  is  terminated  when,  on  inspection,  the 
state  is  found  to  exceed  some  fixed  value  M . In  Model  II  the  process 
is  repaired  (reverts  to  state  0)  when  the  state  at  inspection  exceeds 
M . Simple  conditions  are  given  which  imply  that  the  optimal  inspec- 
tion interval  is  a non-increasing  function  of  the  last  observed  state. 

Model  III  is  an  inventory  process  with  uncertain  supply  as  well  as 
demand.  Given  order  size  n , the  number  received  is  Binomial  (n;p)  . 

^ Costs  of  ordering,  storage,  and  shortage  are  incorporated.  In  the 

single  period  case,  conditions  are  given  which  imply  that  the  optimal 
j order  size  is  non-increasing  in  current  inventory.  This  result  extends 

to  the  undiscounted  multiperiod  case  provided  the  holding  cost  is  zero, 
i A counterexample  is  given  for  a two  period  case  with  linear,  non-zero 

[■’  holding,  shortage,  and  ordering  costs. 
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MARKOV  INSPECTION  PROCESSES: 
INTRODUCTION  AND  APPLICATIONS 


1.1  Introduction 

We  will  be  ccncerned  with  processes  for  which  the  state  is  not 
known  with  certainty  unless  a specific  act  of  inspection  is  performed. 
Once  the  state  is  determined,  a maintenance  action  is  chosen  according 
to  a policy  which  is  fixed  in  ddvar.ce.  Whether  or  not  the  process  is 
inspected.  It  undergoes  a random  transition  into  a new  state.  Thus, 
at  each  time  period,  a choice  must  be  made:  to  operate  the  process 

without  inspection,  or  to  inspect,  and  possibly  maintain  the  process. 

A variety  of  problems  which  have  appeared  in  the  literature 
fit  this  model.  The  applications  we  will  consider  fall  under  three 
main  headings:  terminating  processes,  repairable  processes,  and  an 

inventory  model  with  uncertain  production.  In  this  chapter  we  will 
discuss  briefly  each  of  these  models,  and  give  applications.  In 
Chapter  2,  a common  mathematical  framework  for  these  models  is 
developed.  We  prove  a general  theorem  about  the  structure  of  optimal 
policies.  In  Chapters  3 and  4 we  explore  in  detail  the  models  pre- 
sented in  this  chapter,  and  apply  the  results  of  Chapter  2. 

1.2  A Brief  Summary  of  the  General  Model 

Our  underlying  process  is  discrete  in  time,  and  its  state  can  be 
described  by  an  integer.  In  general  the  state  is  not  known. 

If  the  process  is  not  inspected,  a state  dependent  cost  is  in- 
curred. The  process  uncergoes  a random  transition  according  to  a 
transition  probability  matrix  ? . The  state  remains  unkno^-n. 


I 


2 : 
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If  the  process  is  inspected,  again  a state  dependent  cost  is 

I 

i 

incurred.  The  process  undergoes  a transition  according  to  a 
(possibly  different)  probability  matrix  Q . Thus,  the  act  of 
inspection  may  affect  the  underlying  process.  In  addition,  under 
inspection  the  next  state  of  the  process  becomes  known. 

We  shall  assume  throughout  that  the  process  is  Markovian. 

This  means,  given  the  present  state  and  decision,  that  future 
states  (and  costs)  are  conditionally  independent  of  past  states 
and  decisions. 

The  process  described  above  will  be  called  a Markov  Inspection 
process.  For  a more  formal  definition,  see  Chapter  2. 

We  will  be  concerned  with  characterizing  the  structure  of  policies 
which  minimize  the  expected  total  discounted  cost  (or  average  cost, 

where  appropriate)  of  operating  the  process.  | 

1.3  Model  I:  Terminating  Inspection  Processes  * 

In  this  model  we  will  assume  that  the  process  has  a finite  | 

state  space  (0,1,  ...,  N)  . When  inspection  occurs  and  the  process  ( 

i 

is  discovered  to  be  in  a state  j which  exceeds  some  critical  value 
M , the  process  terminates. 

The  terminating  process  model  may  be  applied  to  the  problem  i 

of  minimizing  the  operating  cost  of  a non-repairable  machine.  We  | 

suppose  the  machine  can  be  in  any  one  of  N + 1 states  ranging 
from  0 (perfect)  to  N (failed).  Transitions  occur  according 
to  a Markov  chain  and  the  state  remains  unknown  unless  the  machine 
is  inspected.  The  cost  of  inspection  depends  on  the  state,  and 


there  is  a cost  per  period  for  undetected  failure. 


In  Chapter  3 we  give  conditions  such  that  the  optimal  number 
of  periods  between  successive  inspection  is  a nonincreasing  function 
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k. 


of  the  state  at  last  inspection. 

A second  example  of  a terminating  inspection  process  is  the 

"optimal  reject  allowance"  problem.  In  its  simplest  formulation, 

a shop  must  fill  an  order  for  N items.  It  costs  C + kC  to 

o 

produce  a lot  of  k items.  Each  item  has  a fixed  probability  q 

of  being  defective,  independently  of  the  others  in  the  lot.  Thus 

the  number  of  usable  items  resulting  from  a lot  of  size  k has 

distribution  Binomial  (k;l  - q)  . The  problem  is  to  compute  the 
* 

optimal  lot  size  k (N)  , or  equivalently,  the  optimal  reject  allow- 
ance  k (N)  - N . This  problem  is  studied  in  detail  by  A.  Beja  [ 1 ] . 

We  may  consider  Beja's  model  as  a terminating  inspection  process 
as  follows.  The  state  is  the  number  of  good  items  on  hand,  or  N , 
whichever  is  smaller.  If  k items  of  the  present  lot  have  been 
produced  but  not  inspected,  we  may  produce  another  item  without 
inspection,  or  terminate  production  and  determine  the  number  of  good 
items.  The  latter  action  corresponds  to  inspection.  If  the  state, 
after  inspection,  is  less  than  N , a new  lot  must  be  started  and  a 
set-up  cost  is  incurred.  (This  corresponds  to  an  inspection 

cost  in  our  general  model  of  Chapter  2).  If  the  state,  after  inspec- 
tion, is  N , the  process  terminates. 

* 

Beja  proves  that  the  optimal  lot  size  k (N)  is  nondecreasing 
in  N . He  also  shows  that,  if  f (k)  is  the  cost  of  first  producing 
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f(k  + 1)  f (k)  for  k < k (N) 

and 

f(k  + 1)  ^ k(k)  for  k ^ k (N)  . 

These  results,  besides  being  of  theoretical  interest,  have 
computational  implications  which  are  useful  in  finding  optimal 
policies. 

In  Chapter  3,  using  our  Markov  inspection  process  formulation, 
we  will  extend  these  results  to  the  case  where  the  output  from  a 
lot  of  size  k is  determined  by  k transitions  of  a Markov  chain. 
This  includes  the  binomial  case,  the  batch  production  case,  and,  in 
particular,  the  case  where  the  number  of  usable  items  has  the  Poisson 
distribution  with  parameter  kX  . 

1.4  Model  II;  Inspection  Processes  with  Repair 

This  model  differs  from  the  terminating  inspection  model  in 
that,  when  an  inspection  takes  place  and  the  state  is  discovered 
to  exceed  a critical  value  M , the  process  is  repaired  and  reverts 
to  state  0 instead  of  terminating. 

A motivating  example  is  the  production  process  described  by 
Sackrowitz  and  Samuel-Cahn  [11],  in  which  the  states  of  successive 
units  produced  form  a Markov  chain.  A unit  may  be  inspected  at 
fixed  cost,  and,  if  defective,  may  be  replaced  by  a perfect  item. 

If  not  inspected,  the  item  becomes  part  of  the  output  process  without 
being  observed.  Under  the  assumption  that  the  Markov  chain  is 
irreducible,  the  authors  prove  that  a stationary  inspection  policy 
is  optimal.  That  is,  if  the  last  item  inspected  was  in  state  i , 
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it  Is  optimal  to  produce  a fixed  number  t^  of  items  before 
inspecting  again. 

In  the  present  work  we  are  assuming  that  the  process  cannot 
improve  (decrease  in  state)  spontaneously.  We  feel  this  is  more 
realistic  than  the  assumption  of  irreducibility . When  state  N 
is  reached,  and  discovered  via  an  inspection,  the  process  is  repaired 
and  reverts  to  state  0 . In  Chapter  3 we  will  present  conditions 
under  which  the  optimal  inspection  intervals  t^  are  nonincreasing 
in  i . 

Much  work  has  been  done  on  the  related  three  action  model 
where  no  action,  inspection,  and  repair  are  independent  alternatives. 

In  particular,  S.  Ross  [10]  considered  the  observed  state  space  to 
be  {P  = (Pq,P^,  ...)  : P^  ^ 0 , ^ P^  = 1}  with  the  interpretation 
that  P^  is  the  posterior  probability  that  the  underlying  process 
is  in  state  i . He  proves  that  the  optimal  inspect  and  repair 
regions  are  convex.  For  the  two-state  problem  this  implies  a 
four  region  structure  for  the  optimal  policy:  Take  no  action  if 

0 ^ p < p^  , inspect  if  p^  ^ p < P2  , take  no  action  if  P2  P P3  > 

and  repair  if  P^  P i ^ • (Here  p is  the  probability  of  being 
in  the  failed  state). 

In  our  model  there  will  be  only  one  critical  number  for  the 
two-state  case:  if  p < p^^  take  no  action;  if  P ^ Pj^  inpsect 

(and  repair  if  necessary) . 

Ross'  work  was  generalized  by  D.  Rosenfield  [ 8 ],  who  presents 
a model  in  which  the  optimal  policy  can  be  characterized  by  three 
critical  numbers  for  each  underlying  state.  Under  specified  condi- 
tions on  the  costs  and  transitions  matrix,  Rosenfield  proves  that. 


6 

if  the  last  known  state  was  i and  k transitions  have  since 
occurred,  it  is  optimal  to  take  no  action  if  0 ^k  < k^(i)  , 
inspect  if  k^(i)  ^ k < k2(i)  , take  no  action  if  k2(i)  ^ k < k^d)  , 
and  repair  if  k ^ k^Ci)  . Two  or  more  of  the  critical  numbers  may 
coincide  for  a given  i . In  addition,  he  shows  that  k^(i)  and 
k^(i)  are  nonincreasing  in  i . 

Under  conditions  similar  to  those  given  by  Rosenfield,  we  will 
show  in  Chapter  3 that  an  analogous  result  holds  for  our  two  action 
model.  That  is,  the  optimal  policy  is  specified  by  a single  non- 
increasing sequence  of  critical  numbers  k(i)  . It  is  optimal  to 
take  no  action  if  k < k(i)  and  inspect  (and  repair  if  necessary) 
if  k ^ k(i)  . 

We  feel  that  the  present  model  has  several  advantages  over 
Rosenfield ' s . In  our  model  repair  must  be  preceded  by  inspection. 

This  is  realistic  in  many  physical  situations.  Consider,  for  example, 
a complex  system  of  components  operating  in  a "black  box."  Suppose 
that,  once  the  box  is  opened,  the  state  of  the  system  becomes  known. 

If  it  is  necessary  to  open  the  box  in  order  to  repair  the  system,  then 
the  present  model,  rather  than  Rosenfield ' s , is  applicable. 

Furthermore,  our  results  are  valid  in  somewhat  greater  generality. 
Rosenfield  assumes  that  the  cost  of  inspection  is  a constant  surcharge 
over  the  cost  of  non-inspection.  We  need  only  a weaker  assumption 
that  the  cost  of  non-inspection  is  nondecreasing  relative  to  the  cost 
of  inspection  as  the  state  increases. 

Finally,  since  our  inspection  repair  process  is  a two  action 
problem  rather  than  a three  action  problem,  optimal  policies  are  much 
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h 


.a 
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easier  to  compute  and  administer. 
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1.5  Model  III:  An  Inventory  Process  with  Uncertain  Production 

In  this  model  random  demands  for  a commodity  occur  over  a 
number  of  periods.  The  problem  is  to  minimize  the  expected  dis- 
counted sum  of  production,  storage,  and  shortage  costs.  As  stated 
thus  far,  this  is  the  usual  dynamic  inventory  model  (see  for 
example  [ 5 ] ) . 

The  distinctive  feature  of  this  mode]  is  that  the  output  from 
a given  production  lot  or  order  is  random.  The  simplest  case  is 
the  binomial  case  where,  when  n items  are  produced  the  output  of 
usable  items  is  distributed  Binomial  (n,p)  for  some  fixed  p . 

More  generally,  the  model  includes  the  case  of  random  batch  produc- 
tion and,  in  particular,  the  case  of  Poisson  production. 

We  assume  a linear  cost  of  production  (no  set-up  cost).  A 
storage  cost  h(t)  is  imposed  in  each  period  for  t units  of 
inventory  in  excess  of  demand,  and  a shortage  cost  k(t)  Is  imposed 
for  t units  short  of  demand.  Unsatisfied  demand  is  carried  over 
(backlogged)  to  the  next  period. 

A similar  model  was  considered,  for  the  one  period  case,  by 

Karlin  [ ^ ] . Under  the  assumptions  that  the  holding  and  shortage 

costs  are  convex,  he  proves  that  the  optimal  production  level 
* 

n (x)  is  nonincreasing  in  current  inventory  x . (Intuitively  - 
the  less  stock  on  hand,  the  more  should  be  produced). 


However,  Karlin's  approach  does  not  extend  to  the  multiperiod 


case.  The  reason  is  that  the  n period  optimal  cost  function  need 
not  be  convex.  A counterexample  is  given  in  Chapter  4. 
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In  Chapter  A we  reprove  and  extend  Karlin's  result,  without 
assuming  convexity  but  with  some  restriction  on  the  distribution 
of  demand  (assumed  to  be  P?2  , see  [ 6 ] ) . 

We  proceed  by  formulating  the  problem  as  a Markov  inspection 
process  whose  state  is  the  current  inventory.  The  production 
decision  is  regarded  as  being  made  sequentially.  Thus,  suppose 
t items  (or  batches)  have  been  produced  in  the  current  period,  but 
the  output  of  usable  items  has  not  been  determined.  Either  an 
additional  item  (batch)  may  be  produced,  or  production  for  the 
period  may  be  terminated  and  demand  satisfied.  The  latter  action 
corresponds  to  inspection,  in  our  general  model,  and  the  former, 
to  non-inspection. 

Applying  the  theory  developed  in  Chapter  2,  we  prove  that 
Karlin's  intuitive  result  extends  to  the  multiperiod  case,  provided 
there  is  no  storage  cost  and  no  discounting. 

When  there  is  a storage  cost  or  discounting  the  result  need  not 
hold.  In  Chapter  A we  present  an  example  of  a two  period  problem 
with  constant  demand  and  linear  production,  storage,  and  shortage 
costs  for  which  monotonicity  of  the  optimal  production  level  is 
violated.  In  this  example  it  is  optimal  to  produce  one  item  when 
initial  inventory  is  three  items,  but  optimal  production  is  zero 
when  initial  stock  is  two  items.  Thus,  our  intuition  may  fail  even 
in  a very  simple  multiperiod  problem. 

In  the  case  where  there  is  no  storage  cost,  we  are  able  to  obtain 
the  result  that,  if  it  is  optimal  to  produce  when  inventory  is  x 
and  L periods  remain,  then  it  is  optimal  to  produce  when  inventory 
is  X and  L'  periods  remain,  L'  •»  L . 


We  are  unable  to  obtain 


the  stronger  result  that  the  optimal  production  level  is  nondecreasing 
in  the  number  of  periods  remaining.  This  seems  to  be  an  open 
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question. 

For  the  one  period  case  with  linear  holding  and  shortage  costs 
and  Binomial  (n;p)  production,  we  show  that  the  optimal  lot  size 
for  a given  initial  inventory  is  nonincreasing  in  the  unit  ordering 
and  holding  costs  and  nondecreasing  in  the  unit  shortage  cost.  The 
optimal  cost  is  nonincreasing  as  a function  of  p . 
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CHAPTER  2 

MARKOV  INSPECTION  PROCESSES:  THE  GENERAL  MODEL 


2.1  Description  of  the  Model 

Assume  that  the  state  of  a process  at  time  n Is  given  by  an 

integer  X . In  general,  we  will  assume  that  X is  not  knovTi 

n n 

with  certainty.  What  is  known  about  the  process  can  be  expressed 

by  the  ordered  pair  Y = (i,t)  , where  i = X is  the  most 

n n-t 

recent  state  known  with  certainty  and  t periods  have  elapsed 
since  the  state  was  last  determined. 

In  any  observed  state  (i,t)  there  are  two  possible  actions: 

action  0:  non- inspection 

action  I:  inspection. 


Let  A denote  the  action  taken  at  time  n . 
n 

Under  action  0 the  process  undergoes  a transition  according  to 


the  transition  probability  matrix  P . Thus  ^ ^ 


A^  = 0)  = P^j^  independent  of  the  history  of  the  process.  In  addition, 

if  X = k , a cost  0,  > 0 is  incurred  and  no  new  information 
n+1  k — 

about  the  process  is  obtained.  Thus  if  Y = (i,t)  and  A = 0 , 

n ’ n 

the  process  undergoes  t + 1 consecutive  transitions  according  to 


P between  i and  = j , so  that  “ J I ~ » 

A = 0)  = Pj^^  . The  expected  cost  incurred  will  be  C(Y  ,A  ) = 

n 1]  n n 

r t+1 

C(i,t;0)  = I P^^  Oj  . Since  no  information  is  gained,  the  new 

observed  state  will  be  Y = (i,t  + 1)  . 

n+i 
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Under  action  I the  process  undergoes  a transition  according 

to  the  transition  probability  matrix  Q . Thus  ^ ^ ^n  ' ^ ’ 

= I)  = , and  again  is  conditionally  independent  of 

the  histo'’y  of  the  process,  given  X and  A . If  X , , » k , 

n n n+i 

a cost  I,  > 0 is  incurred.  Moreover,  the  new  state  X ,,  becomes 
k — n+1 

known.  Thus  we  interpret  action  I as  inspection.  If  = (i,t) 

and  A = I , the  process  undergoes  t consecutive  transitions 
n 

under  P , followed  by  a transition  under  Q , between  states 

X ^ = i and  X ^,  = j . Thus  P(X  , = 1 I Y = (i,t)  and 

n-t  n+1  n+1  ' n ' > ' 

A^  = I)  = ^ ^ik^kj  ■ follows  that  under  action  I the  expected 

cost  incurred  will  be  C (Y  ,A  ) = C (i , t ; I)  = ^ P^, Q,  I.  ; and  the 

n n , , ik  kj  j 


next  observed  state  will  be  Y 


n+1 


(j,0)  with  probability 


Definition: 

We  will  call  such  a process  a Markov  inspection  process. 

As  will  be  seen  in  subsequent  chapters,  the  models  presented 
in  Chapter  1 are  all  examples  of  Markov  inspection  processes. 

2.2  Definition  and  Existence  of  Discounted  Cost  Optimal  Policies 

We  will  study  the  problem  of  minimizing  the  expected  discounted 
total  cost  of  the  Markov  inspection  process  described  in  2.1. 

We  will  assume  that  there  is  no  upper  bound  to  the  number  of 
non-inspections  (actions  0)  between  two  successive  inspections 
(actions  I),  but  the  process  terminates  after  L(<_  =»)  inspections 
take  place. 


r 
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In  some  applications,  e.g.  models  I and  II  of  Chapter  1,  a 
fixed  period  of  time  elapses  between  transitions.  In  this  case 
future  costs  are  discounted  at  a rate  a ^ 1 per  transition. 

In  other  applications,  e.g.  our  inventory  model  (model  III 
of  Chapter  1) , transitions  involving  non-inspectioi.  are  instantaneous 
and  one  time  unit  passes  between  successive  inspections.  In  this 
case  future  costs  are  discounted  at  a rate  6^1  per  inspection. 

In  the  remainder  of  this  section  we  will  treat  both  cases 
simultaneously.  On  our  applications,  however,  either  a = 1 or 
g = 1 . Note  that  we  are  also  considering  the  undiscounted  case 
a = 6 = 1 . 

Let  TT  be  a policy,  i.e.  any  rule  for  choosing  actions.  Let 

Y (tt)  be  the  observed  state  at  time  n , under  policy  n , Let 
n 

A (it)  be  the  action  chosen  by  it  at  time  n . Let  (it)  be  the 
n X. 

number  of  transitions  under  ' up  to  the  H-th  inspection.  Let 
T^(tt)  = 0 and  fix  a , 3 e (0,1]  . 

Define  the  expected  discounted  cost  of  starting  in  = (i,t) 

and  operating  under  r to  be 


(2.2.1) 


V(i,t,L,TT)  = 


E 

IT 


L-1 

I 6^  I a''c(Y  (tt),A  (tt)) 

1=0  n=T  (tt)+1 


Y 

o 


(i,t) 


where  denotes  the  expected  value  with  respect  to  the  distributions 

of  Y (tt)  , A (it)  , and  T.(t')  induced  by  tt  . Note  all  costs  are 
n n Jt 

nonnegative  so  that  E^  is  well  defined. 

* 


Let  V (i,t,L) 


inf  V(i,t,L,TT)  . 
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Definition: 


* 

A policy  IT  is  optimal  if 


V(i,t,L,TT*) 


= V*(i,t,L) 


for  all  i 


t and  L . 


We  are  now  ready  to  state  a theorem  on  the  existence  of  optimal 
policies . 

Theorem  2.2: 

There  exists  an  optimal  policy  tt  such  that  ^ 

deterministic  function  of  the  current  observed  state  Y (tt)  and  the 

n 

number  of  inspection  opportunities  remaining,  L . That  is, 

A = f(Y  (tt),L)  . 
n n 

* 

Furthermore  V (i,t,L)  satisfies  the  following  optimality 
equation: 


(2.2.2) 


V (i,t,L)  = min  ! 

^ ^ij^jk^^k  «6V*(k,0,L  - 1)]| 


and  f(i,t,L)  may  be  any  function  such  that  f(i,t,L)  = 0 when 
the  first  term  in  (2.2.2)  is  minimal  and  f(i,t,L)  = I when  the 
second  term  is  minimal. 

If  a < 1 , and  the  sequences  ^ ^ j ^ bounded,  then 
(2.2.2)  determines  V (i,t,L)  uniquely. 


Proof : 


The  last  result  is  a standard  theorem  in  discounted  dynamic 
programming  [ 2 ].  The  remainder  of  the  theorem,  which  holds  even 
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when  L = “>  , and  a = S = 1 , follows,  since  all  costs  are  non- 
negative, from  standard  results  in  negative  dynamic  programming  [12]. ■ 

Remark: 

The  first  term  in  the  optimality  equation  (2.2.2)  is  the  expected 
cost  of  the  policy:  perform  action  0,  then  proceed  optimality. 

The  second  term  is  the  expected  cost  of  the  policy  (which  we  will 
call  ■'^q)  : perform  action  I,  then  proceed  optimality.  Thus  the 

optimal  action  is  that  action  which  yields  the  minimum  cost,  when 
all  subsequent  actions  are  optimal.  This  is  just  the  well  known 
principle  of  optimality. 

2.3  A Structure  Theorem 

In  this  section  we  will  prove  a general  result  which  will  apply 
to  the  models  presented  in  Chapter  1. 

We  begin  with  some  definitions. 


Definition: 


A policy  m is  said  to 

be 

monotone 

if. 

for  each 

L , there 

exists  a nonincreasing  sequence 

t*  (i,L) 

such  that 

(I  if 

Y 

=■•  (i,t) 

and 

A 

t >_  t 

(i,L) 

A (it)  = J 

n 

ft 

lo  if 

Y 

n 

- (i,t) 

and 

t < t 

(i,L) 

when  L inspection  opportunities  remain. 

Let  tTq  be  the  policy:  perform  action  I,  then  proceed 


i 


optimally. 


i 


Let  TT^  be  the  policy:  perform  action  0,  then  proceed  according 

to  TTq  . 

Definition : 

A policy  TT  is  said  to  be  a one  stage  look-ahead  policy  if 

A (tt)  = I if  and  only  if  V(Y  ) < V(Y  i.e, 

n n U n 1 

TT  inspects  whenever  immediate  inspection  is  cheaper  than  non- 
inspection followed  by  inspection,  subsequent  actions  being  optimal. 

Remark: 

The  terminology  "one  stage  look-ahead  policy"  comes  from  the 
theory  of  optimal  stopping  problems  [ 9 ] . In  the  present  case  we 
identify  action  I with  "stopping"  and  paying  a terminal  cost  equal 
to  the  expected  cost  of  acting  optimally  for  the  remainder  of  the 
process.  Action  0 is  identified  with  "continuing."  A one  stage 
look-ahead  policy  stops  exactly  when  stopping  is  less  costly  than 
continuing  one  more  stage  and  then  stopping. 

Conditions  for  a one  stage  look-ahead  policy  to  be  optimal  can 
be  found  in  [ 9 ] . 

The  purpose  of  this  section  is  to  give  sufficient  conditions 
for  the  existence  of  an  optimal  policy  that  is  a monotone,  one  stage 
look-ahead  policy. 

We  will  need  to  make  the  following  assumptions  on  the  transition 


matrix  P : 
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Assumpcions : 

Al : P is  upper  triangular,  i.e.  P..  = 0 for  j < i ; and 

^ J 

P^^  < 1 . (If  the  number  of  real  states  is  finite 

(N  + 1)  we  allow  P._,  = 1)  . 

Nl'i 

A2;  P is  TP2  (totally  positive  of  order  2),  i.e. 

^ij^i'j'  “ ^ij'^i'j  ~ ° all  i<i',j<j'. 

Assumption  Al  means  that  the  real  state  of  the  process  cannot 
decrease  under  action  0,  and,  with  positive  probability,  may  increase. 
Assumption  A2  is  a technical  condition  which  implies  among  other 
things  that  P satisfies  the  increasing  failure  rate  (IFR)  property 
[ 3 1,  i.e.  P(X  > i I X ) is  an  increasing  function  of  X for 
each  j . The  TP2  property  is  needed  in  the  proof  of  Lemma  2.3.1 
to  follow.  A complete  discussion  of  total  positivity  is  given  in  [ 6 ] . 

We  will  also  require  the  following 

Structural  Conditions: 

(SCI):  There  exist  critical  numbers  such  that  V(J,0,L,iTq)  ^ 

V(j,0,L,TT^)  for  j < V(j,0,L,TTQ)  < V(J,0,L,t^) 

for  j ^ jj  • (Thus  if  the  true  state  is  known  to  be 
j ^ , then  immediate  inspection  is  less  costly  than 

waiting  one  period  and  inspecting) . 

(SC2) : For  each  (i,t)  there  exists  t' ^ t such  that  inspection 

is  strictly  optimal  in  state  (i,t')  , when  L inspection 
opportunities  remain. 


I 


1 


r 


I 
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The  above  structural  conditions  will  be  verified  for  each  of 
the  models  presented  in  Chapter  1,  in  succeeding  chapters. 

We  are  now  ready  to  state  the  main  theorem  of  this  chapter. 

Theorem  2.3: 

If  assumptions  A1  and  A2  are  satisfied,  and  the  structural 
conditions  (SCI)  and  (SC2)  hold,  then  a monotone,  one  stage  look- 
ahead policy  is  optimal. 

Before  beginning  the  proof,  we  will  need  one  more  definition 
and  a lemma. 

Definition: 

A sequence  is  said  to  have  the  single  crossing  property 

(SCP)  if  > 0 implies  A^^  > 0 for  all  k > j . 

Remark: 

The  usual  definition  is  somewhat  weaker:  A . > 0 =*■  A,  >0 

J k - 

for  k > j . This  is  not  sufficient,  without  further  argument,  to 
ensure  monotonicity  of  the  critical  numbers.  See  the  appendix  to 
this  chapter  for  further  discussion  and  a proof  of  the  following 
lemma . 


Lemma  2.3: 

If  Aj  satisfies  the  SCP  in  j 

then  y P^.A^  satisfies  the  SCP  in 
^ ij  j 


I 

and  P satisfies  Al  and  A2,  j 

i and  t . 3 


Proof : 


See  the  appendix  to  this  chapter.  ■ 
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Proof  of  Thftorem  2.3: 

Defining  A(i,t,L)  = V(i,t,L,n^)  - V(i,t,L,Tip)  and  conditioning 

on  the  true  state  X given  Y = (i,t)  yields: 

n n 

(2.3.1)  A(i,t,L)  = I pJ^A(j,0,L)  . 

(This  key  result  may  also  be  seen  algebraically  by  direct  expansion 
of  V(i,t,L,7TQ)  and  V(i , t ,L , tt^)  ) . 

The  first  structural  condition  (SCI)  is  equivalent  to 
A(j,0,L)  1 0 , j < and  A(j,0,L)  > 0 for  j 1 jj^  , i.e.  A(j,0,L) 
has  the  SCP  in  j . 

It  immediately  follows  from  Lemma  2.3  and  (2.3.1)  that  A(i,t,L) 

I has  the  SCP  in  i and  t . 

I 

< * 

We  can  now  define  the  critical  numbers  t (i,L)  = 

min  {t  : A(i,t,L)  > 0}  . 

By  (SC2) , for  some  t^  , inspection  is  strictly  optimal  in 

* 

(i,t^)  , which  implies  that  A(i,t^,L)  > 0 . Hence  t (i,L)  < “=  . 

"k 

By  the  SCP  in  t , A(i,t,L)  £ 0 for  t < t (i,L)  and 

A(i,t,L)  > 0 for  t ^ t (i,L)  . 

Furthermore,  since  A(i,t,L)  satisfies  the  SCP  in  i , we  must 
* 

conclude  that  t (i,L)  is  nonincreasing  in  i . (Otherwise,  if 

k k 

t (i  + 1,L)  > t (i,L)  , then,  for  some  t , A(i,t,L)  > 0 and 

A(i  + l,t,L)  ^ 0 , contradicting  the  SCP). 

Let  us  define  the  policy  by  ^ only  if  L 

★ 

inspection  opportunities  remain,  Y = (i,t)  , and  t > t (i,L)  . 

n — 

k k 

Then  i:  is  a monotone  policy,  tr  is  also  a one  stage  look-ahead 

* 

. policy,  since  t inspects  in  state  (i,t)  if  and  only  if 

1 

[ 

: 


\ 
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immediate  inspection  is  less  costly  than  non-inspection  followed 

by  inspection,  subsequent  actions  being  optimal. 

* 

It  remains  only  to  prove  that  tt  is  optimal. 

Let  t < t (i,L)  . Then  V(i,t,L,Tt^)  £ V(i,t,L,TtQ)  . 

The  term  in  the  optimality  equation  (2.2.2)  corresponding  to 
action  0 is 

y P..  0.  + aV  (i,t  + 1,L)  < y P:.  0.  + aV(i,t  + 1,L,tt„)  = 

^ ij  j - ^ ij  j ’ ’ 0' 

V(i,t,L,TT^)  V(i,t,L,rQ)  , 

which  is  the  term  of  (2.2.2)  corresponding  to  inspection.  It 

follows  from  Theorem  2.2,  that  action  0 is  optimal  in  state  (i,t) 

★ 

when  t < t (i,L)  . 

★ 

We  next  consider  the  case  t ^ t (i,L)  . Suppose,  to  obtain 

a contradiction,  it  is  strictly  optimal  to  perform  action  0 when 

* 

Y = (i,t')  , t'  > t (i,L)  . Let  t''  = rain  {t  > t'  ; action  I 
n — 

is  optimal  in  (i,t)}  . (The  above  set  is  non-empty  by  (SC2)). 

Note  that  action  0 is  optimal  in  (i,t''  - 1)  and  action  I is 
optimal  in  (i,t'')  . This  implies  that  is  an  optimal  policy 

in  state  (i,t''  - 1)  , hence  V(i,t''  - l,L,ir^)  £V(i,t''  - l,L,iTp)  . 

■k 

But  t''  - 1 ^ t (i,L)  so  A(i,t''  - 1,L)  > 0 , yielding  a con- 

•k 

tradiction.  It  follows  that  action  I is  optimal  for  all  t ^ t (i,L)  . 

★ 

Thus  we  have  shown  that  ■:t  is  optimal,  and  the  proof  is 
complete.! 
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Appendix  to  Chapter  2: 

One  of  the  major  properties  of  TP2  matrices  is  that  they 
preserve  the  single  crossing  property.  In  this  appendix  we  will 
use  the  term  weak  single  crossing  property  (WSCP)  for  the  usual 
definition,  with  weak  inequality  instead  of  strict  inequality. 

(See  the  remark  following  the  definition  of  the  single  crossing 
property) . 

Karlin  [ 6 ] proves  that  if  has  the  WSCP  in  j , then 

y P. .A.  has  the  WSCP  in  i . In  [ 7 ] Rosenfield  shows  that  if 
^ ij  1 

in  addition  to  being  TP2  (Al) , P is  upper  triangular  (A2)  then 

y P^,A.  has  WSCP  in  i and  t . 

^ ij  J 

We  wish  to  extend  these  results  to  the  strong  version  (SCP) 
defined  above. 

Let  A(j)  have  the  SCP  and  P satisfy  Al  and  A2 . Let 
A(i,t)  = y • Suppose  ACi^.t^)  > 0 , and  ^ ^0  ’ 

By  Rosenfield's  lemma,  A(iQ,tj^)  ^ 0 . Suppose,  to  obtain  a con- 
tradiction, A(iQ,t^)  = 0 . Let  0 < e < A(iQ,tQ)  and  let 
i A(j)  - e if  A(j)  £ 0 

A'(j)  =\  . Then  A'(j)  satisfies  the  SCP. 

(A(j)  if  A(j)  > 0 

Letting  A'(i,t)  = y , we  have  A(i,t)  - e £ A'(i,t)  £ A(i,t) 

Hence  A'(iQ,tQ)  > 0 . Now  A'(i,t)  satisfies  the  WSCP  in  t so 
that  A’(iQ.t^)  2^0  . But  A'(iQ,t^)  < A(iQ,t^)  = 0 , yielding  a 
contradiction.  Thus  A(i,t)  has  the  SCP  in  t . The  same  proof 
shows  that  A(l,t)  has  the  SCP  in  i . This  completes  the  proof 
of  Lemma  2.3.1 
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CHAPTER  3 


MODELS  OF  INSPECTION  FOR  TERMINATING  AND  REPAIRABLE  PROCESSES 

« 

3.1  INTRODUCTION 

In  this  chapter  we  will  characterize  the  structure  of  optimal 
policies  for  the  terminating  and  repairable  processes  (models  I and 
II)  introduced  in  Sections  1.3  and  1.4 

We  will  assume  that  the  underlying  process  has  a finite  state 
space  (0,1,  ...,  N}  . When  inspection  takes  place  and  the  state 
is  observed,  termination  or  repair  occurs  according  to  a policy 
which  is  fixed  in  advance.  This  policy  will  be  assumed  to  have  the 
following  form:  Terminate  (repair)  the  process  when  the  observed 

state  j exceeds  some  critical  value  M N . Otherwise  the  process 
is  not  altered  by  inspection. 

The  repairable  model  fits  our  general  scheme  of  Chapter  2 as 

follows:  Under  non-inspection  the  transition  probability  is 

P(X  ,1  = i I X = i,A  = 0)  = P . . as  in  Chapter  2.  Under  Inspection, 
n+1  ' n n ij 

the  transition  probability  matrix  Q is  related  to  P by 


= P(X 


N 

I P-- 

j=M  "J 


iH 


= j i X_  = i,A_  = I)  =<P^. 


j = 0 
1 j < M 
j ^ M . 


Thus,  under  Inspection  the  process  is  returned  to  state  0 if  it 
would  otherwise  have  been  in  a repairable  state  (^  M)  after  a 
transition  according  to  P . For  j ^ M the  inspection  cost  Ij 
is  considered  to  include  the  cost  of  repair. 
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Remark : 

Under  our  assumptions  on  the  costs  and  transition  matrix,  if 

the  underlying  state  j were  known  at  all  times,  the  optimal  repair 

policy  would  have  the  structure  given  above.  That  is,  it  is  optimal 

to  repair  exactly  v^heu  j ^ M for  some  M . More  general  sufficient 

conditions  for  this  type  of  policy,  called  a "control  limit  policy" 

to  be  optimal,  are  given  by  Uerman  [ J ]. 

To  show  that  the  terminating  process  also  fits  our  general  model, 

* 

we  will  define  a terminated  state  N . The  process  enters  state 
* 

N only  under  inspection  when  the  state  would  otherwise  have  been 

^ .M  under  a transition  according  to  P . Thus  the  transition  matrix 

P is  defined  as  in  Chapter  2 bv  P . . = P (.X  , = i I X = i,A  =0) 

ij  n-*l  ' n ’ n 

with  the  additional  conditions  that  P = 0 , i < N and  P * * ~ ^ • 

iN  ~ N N 

The  transition  matrix  Q for  inspection  is  given  by 


for  i . 3nJ  Q * * ~ 1 further  define  the  inspection  cost 

N N 

I ^ and  the  non-inspection  cost  0 ^ to  be  ^ero.  Thus,  when  the 
N X 

process  is  inspected  and  discovered  to  be  entering  a state  j ^ X , 

* 

it  is  placed  in  state  X and  no  further  costs  are  incurred.  The 
inspection  cost  I • < i 2.  “ considered  to  include  any  cost  of 


terminating  the  process. 


In  this  chapter  we  will  consider  the  number  of  potential  in- 
spection opportunities  L to  be  infinite.  We  will  discount  costs 
at  each  transition  (inspection  or  non-inspection)  so  that  ct  ^ 1 
and  6=1. 

In  light  of  the  above  assumptions,  the  optimality  equation 
(2.2.2)  reduces  to 


(3.1)  V*(i,t)  = min  + D ; I 

We  will  show  that  the  optimal  policy  for  both  processes  is 

* 

specified  by  a nonincreasing  sequence  of  critical  numbers  t (i)  , 
such  that  it  is  optimal  to  inspect  in  state  (i,t)  if  and  only  if 
t > t*(i)  . 

3.2  Structural  Results  for  Model  I 

In  order  to  use  the  results  of  Section  2.3,  we  will  again  make 
assumptions  Al  and  A2  of  that  section:  The  transition  matrix  P is 

upper  triangular  with  < 1 for  i < N and  P is  TP2  . 

We  will  need  to  show  that  the  structural  conditions  (SCI)  and 
(SC2)  of  Section  2.3  hold.  We  will  therefore  make  the  following 
assumptions  on  costs: 


Bl: 


B2: 


I.  ^ 0^  > j M , and  I^  - 0^  is  nonincreasing  in  j , 
for  j <_  N . 

’j  ' “j  ^ ^ ■ 


Assumption  Bl  means  that,  for  the  "good"  states  j < M , the 
cost  of  Inspection  is  larger  than  the  cost  of  non-inspection,  but 
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I 


Ir 

k 


L. 


the  the  cost  of  non- inspection  increases  relative  to  the  cost  of 
inspection  as  the  state  becomes  higher. 

In  order  to  interpret  condition  B2,  V3  will  introduce  the 
auxiliary  observable  state  (j,-D  • 

Definition: 

We  will  say  that  = (j,-l)  if  the.  value  of  is  known 

to  be  j before  the  decision  Aq  is  chosen. 

Note  that  this  is  consistent  with  our  definition  of  (j,t)  , 
t ^ 0 . 

Assumption  B2  is  equivalent  to  VCj.-l.TT^)  < V(j,-1,TT^)  or 
A(j,-1)  > 0 , for  j . This  means  that,  given  the  next  state 

will  be  j ^ M , immediate  inspection  and  stopping  is  less  costly 
than  non-inspection  followed  by  immediate  inspection  and  stopping. 

If  inspection  is  strictly  optimal  for  state  (j,-l)  » j 
then  B2  must  hold.  The  reserve  implication  is  also  true,  as  a result 
of  the  proof  of  Theorem  2.3. 

For  the  case  M * N , B2  reduces  to  I^,  < , i.e.  inspection 

N 1 - a 

(and  stopping)  is  cheaper  than  non-inspection  forever. 

Finally,  if  a = 1 , 0^  > 0 , and  1^  is  nondecreasing  in  j , 
j , then  B2  is  satisfied. 

We  are  now  ready  to  state  the  main  theorem  of  this  section. 
Theorem  3.2 : 

Under  the  assumptions  Al,  A2,  B1  and  B2 : There  exist  critical 

★ 

numbers  t (i)  < ® nonincreasing  in  i , i < M , such  that,  defining 

A 

TT  to  be  the  policy  which  inspects  at  (i,t)  if  and  only  if 
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t ^ t (i)  , Ti  is  optimal. 

* * 
Furthermore  n is  a one-stage  look-ahead  policy,  i.e.  tt 

inspects  at  (i,t)  if  and  only  if  inspection  at  (i,t)  is  cheaper 
than  non-inspection,  followed  by  inspection  at  (i,t  + 1)  , sub- 
sequent actions  being  optimal. 

Proof : 

By  Theorem  2.3,  it  is  sufficient  to  establish  the  structural 
conditions  (SCI)  and  (SC2).  We  will  need  the  following  two  lemmas: 

Lemma  3.2.1: 

* r ^”*'0  * 

Information  inequality:  V (i,t)  ^ I V (j,tQ)  for  all 


Proof : 


Let  IT  be  the  policy  "proceed  optimally  under  the  assumption 

that  Yn  = (i,t)  ."  Let  C(tt)  = J (Y  (”)  ,A  (”))  be  the  (random) 
U n n 

* 

total  discounted  cost  under  tt  . Thus  V (i,t)  = V(l,t,’r)  = 

E(C(tt)  1 Yq  = (l,t))  = E(C(tt)  j X ^ = i)  » (using  the  Markovian 


property  of  the  process) 
t-t, 

> 

Ij 


1 P,/V(J,t„)  .1 


Remark: 

We  call  this  the  information  inequality  because  it  states  that, 
if  the  decision  maker  is  informed  of  the  outcome  of  the  (c  - t^) 
step  transition  from  state  1 , and  acts  optimally  on  that  infor- 
mation, this  action  cannot  increase  total  expected  cost. 
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Lemma  3 . 2.2: 

• ’ • ’ ^ and  any  i , 


Proof : 

From  Assumption  Al,  the  states  {0,1,  ...,  N - 1}  are  transient 
and  state  N is  absorbing.  It  follows  that 


(a)  For  any  sequence 
lim  y P^.A,  = 

^ T T 1 XI 


^-►00 


IJ  1 


(b)  lim  V (i,t)  = I 


N 


t (0  j < 

P . = { 

U j-» 


This  proves  (a) . 

To  prove  (b) , let  e > 0 be  given  and  let  T be  sufficiently 
large  that  t ^ T implies  Pj„  > 1 - e . Then  for  t > T , 

V (i,t)  £V(i,t,TT  ) = I P^t^d.  + aV*(j,0))  < Me  + (1  - e)T 

^ ij  J ~ N 

where  M = max  + aV  (j,0)|  is  a constant  independent  of  e . 

* * 

(V  (N,0)  = 0 since  (N,0)  is  identified  with  (N  , 0))  . It 

follows  that 

★ 

lim  sup  V (i,t)  ^ 1 . 

t -»■  00  ^ 

To  obtain  the  reverse  inequality,  we  will  use  Lemma  3.2.1  to 
* _ * 

obtain  V (i,t)  ^ V (j,-l)  . Hence  if  t ^ T , w’e  have 

* * 

V (i,t)  2.  ~ (N»~l)  • Since  N is  an  absorbing  state. 


either  it  is  optimal  not  to  inspect  at  all  future  times,  at  cost 


0, 


N 


1 - 
B2. 


— , or  it  is  optimal  to  inspect  (and  stop)  at  cost  . By 

* 

, so  inspection  is  optimal  and  V (N,-l)  = . 


Thus  T ^ t Implies  V (i,t)  ^ (1  - follows  that 

* 

lim  inf  V (i,t)  ^ . atid  the  result  is  proved. ■ 

t ->  00 


Proof  of  Theorem  3.2  (continued): 

To  establish  (SCI)  we  must  show  that  A(j,0)  has  the  SCP  in  j . 
As  in  the  proof  of  Theorem  2.3,  we  may  express  A(j,0)  = T P A(k,-1)  . 

J K 

By  Lemma  2.3,  if  A(k,-1)  satisfies  the  SCP,  so  does  A(j,0)  . 

Notice  that  A(k,-1)  = V(k,-l,7r^)  - V(k,-l,TiQ)  = + 

a I «V*({,,0))  - (Ij^  + aV*(k,0))  . If  k , then 

V*(2,0)  =0  for  £ ^ k , so  that  A(k,-1)  = 0 + a V PI  - I,  > 0 

by  Assumption  B2. 

Thus,  to  establish  that  A(k,-1)  has  the  SCP,  it  suffices  to 
show  that  A(k,-1)  ^ 0 for  k < M . We  consider  two  cases. 

Case  I: 

* - * 

V (k,0)  = I (£,0))  . (Inspection  is  optimal  in 

observed  state  (k,0))  . In  this  case  A(k,-1)  reduces  to 

Oj^  - Ij^  ^ 0 (by  Assumption  Bl). 

Case  II: 

* r * . 

V (k,0)  = ) • (Non-inspection  is  optimal  in 


L 


State  (k,0))  . In  this  case  A(k,-1)  becomes 
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[o  I - 0,)  - - 0^) 


The  first  term  is  non-positive  from  B1  because  (I.  - 0.)  is  non- 

J J 

increasing  in  j and  - Oj^  ^ 0 . TVie  second  term  is  non-positive 
as  a result  of  Lemma  J.2.1.  Thus  A(k,-1)  ^ 0 for  k < M . The 
first  structural  condition  (SCI)  is  now  estcblished. 


Remark ; 


Assumption  El  can  be  weakened  to 


Bl'  : a y P,  „ (I  - OJ  < I,  - 0,  > 0 , k < M 

^ kH  I I — k k — 


To  establish  the  second  structural  condition  (SC2)  , we  consider 
Che  optimality  equation  (3.1).  Let  be  the  difference  of  the 

terms  corresponding  to  non-inspection  and  inspection,  i.e. 


r t+l  * 

= I P44  0-  + (i,t  + 1) 
it  ^ ij  j 

- I + aV*(j,0))  . 


We  must  show  Chat  for  any  i , for  sufficiently  large  t , inspection 
is  strictly  optimal  when  in  state  (i,t)  . By  Theorem  2.2,  it  will 


be  sufficient  to  show  that  lira  A,  > 0 . Rewriting  A.  as 

it  It 

t-«o 


^it  “ ^ ^ a(v*(i,t  + 1)  - I P^jH'(j.O)) 


we  apply  Lemma  3.2.2  and  obtain 


lira  A.^  ’ (Oj^  - I^)  + a(I^.  - V (N,0)) 

t-«o 


Since  V (N,0)  = 0 , this  becomes 

lim  A.  = 0 + al.,  - > 0 by  B2, 

It  N N N 

t-«o 

I 

Hence  (SC2)  is  established.  By  Theorem  2.3,  the  proof  is  now  j 

1 

complete . ■ 

3.3  Structural  Results  for  Model  II: 

Model  II  (inspection  with  repair)  can  be  viewed  as  a special 

case  of  model  I by  considering  the  optimal  cost  after  repair, 

* 

V (0,0)  , to  be  a terminal  cost,  and  including  it  in  the  inspection- 

A 

repair  cost  • J ^ M . In  order  for  V (0,0)  to  be  finite, 
of  course  in  this  case  we  must  have  a < 1 . 

We  prefer,  for  reasons  of  clarity  to  let  I.  , j include 

•k 

only  the  cost  of  inspection  and  repair,  and  leave  V (0,0)  separate. 

This  will  require  only  minor  modifications  to  the  proof  of  Theorem  3.2. 

We  need  only  modify  Assumption  B2 : 

B2':  I + aV*(0,0)  < 0 . + a J]  + aV*(0,0))  . 

J J j K K 

Theorem  3.3: 

Under  Assumptions  Al,  a2,  B1,  and  B2',  the  conclusion  of  Theorem 
3.2  hold. 

The  condition  B2 ' means  that,  in  state  (j,-l)  , j , 

immediate  inspection  (and  repair)  is  cheaper  than  non- inspect  ion 
followed  by  inspection,  subsequent  actions  being  optimal.  From  the 
proof  of  Theorem  2.3,  condition  B2'  is  equivalent  to  inspection  I 

being  optimal  for  all  (j,t)  , j ^ M , t ^ -1  . 
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* 

The  problem  with  B2'  is  that  it  involves  V (0,0)  , which  is 

not  readily  obtainable  from  the  parameters  of  the  process.  However, 
— * 

if  an  upper  bound  V ^ V (0,0)  is  available,  a sufficient  condition 
for  B2'  is  that  the  inequality  hold,  with  V substituted  for 
v*(0,0)  : 

B2"  : I.  + aV  < 0.  + a y P.,  (I,  + aV)  . 

3 3 ^ jk  k 

Lemma  3.3; 

— * 

If  B2"  holds  with  V^V  (0,0)  , then  B2'  holds. 

Proof : 

Immediate,  since  the  coefficient  of  V is  a on  the  L.H.S., 

2 

and  a on  the  R.H.S.B 


Remark: 


One  such  upper  bound  V can  be  obtained  by  considering  the 

policy  -rr  : "never  inspect."  The  expected  cost  of  acting  according 

to  this  policy  starting  in  state  (0,0)  is  V(0,0,tt)  = 

00 


I 

t*0 


I p 


t+1., 
Oj  3 


★ 

which  is  of  course  not  smaller  than  V (0,0)  . 


1 
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CHAPTER  4 


AN  INVENTORY  MODEL  WITH  RANDOM  PRODUCTION 


4.1  Introduction 


We  consider  the  usual  dynamic  inventory  problem  of  minimizing 
production,  shortage,  and  holding  costs  over  time,  subject  to 
random,  periodic  demands.  (See  for  example  [ 5 ]).  We  impose  the 
added  feature,  however,  that  when  producing  (or  ordering)  the  amount 
actually  added  to  stock  is  random. 

The  simplest  case  of  this  model  is  the  binomial  case.  Suppose 
each  item  produced  has  a fixed  probability  q of  being  unusable, 
independently  of  all  other  items  produced.  It  follows  that  if 
production  level  n is  chosen  the  number  of  usable  items  produced 
will  be  distributed  Binomial  (n,p)  , where  p = 1 - q . 

More  generally,  we  will  also  consider  a model  in  which  produc- 
tion occurs  in  discrete  batches  of  random  size  p . Batch  sizes 
are  assumed  independent  with  a common  distribution  P(p  = j)  = p^  . 

When  production  level  n is  chosen,  the  number  of  usable  items 

11 

produced  is  o(n)  = p . • The  binomial  model  is,  of  course,  a 

1 J 

special  case  of  this  with  p^  = q and  p^  = p . Another  special 
case  is  the  Poisson  model,  in  which  o(n)  is  distributed  Poisson 

X^e"^ 

(nX)  . This  corresponds  to  p^  = — j-j — , j = 0,1,  ...  . 

We  will  assume  that  demands  occur ing  in  each  period  are  in- 
dependent, discrete  random  variables  ^ 0 , with  a common  density 
P(5j^  ' j)  “ • Unsatisfied  demands  will  be  assumed  backlogged 

(carried  over  to  the  next  period).  Thus,  Inventories  in  successive 
periods  are  linked  by  the  equations 


32 


(4.1.1) 


i+1 


X + a - 
111 


Here  x.  is  the  initial  inventory  in  period  i and  v.  = x,  + a, 
i ' ' 1 i i 

is  the  inventory  after  proikiction  but  before  demand  is  satisfied. 

A negative  value  of  signifies  a cumulative  excess  of  demand 

over  production.  Ke  are  implicitly  assuming  that  there  is  no  lag 

in  production,  i.e.  production  undertaken  during  the  current  period 

will  be  available  to  meet  that  period's  demand. 

The  cost  of  producing  at  level  n , c(n)  , is  assumed  to  be 

linear  with  no  set-up  cost,  i.e.  c(n)  = nc  , c > 0 . 

A holding  cost  h(y^  - is  incurred  whenever  the  inventory 

after  production  exceeds  demand.  A shortae;e  cost  k(£i.  - v.)  is 

1 ' 1 

incurred  whenever  there  is  insufficient  inventory  to  meet  demand. 

We  v;ill  assume  that  h(")  and  k(*)  are  nondecreasing  functions 
which  vanish  for  negative  arguments. 

If  initial  inventory  is  x^  and  production  level  n^  is 
chosen,  the  cost  for  period  i will  be 

I 

g(x^,n^,o^,f;^)  = cn^  + h(x^  '*■  " ^i  " • i 


We  seek  a policy  m (a  rule  for 
the  expected  total  discounted  cost  E 


choosing  n.) 

'l-1  „ 

L e=o 


to  minimize 


The  number  of  periods  L may  be  either  finite  or  infinite.  The 
expectation  is  with  respect  to  the  distribution  of  and  the 

distribution  of  ir.diiced  by  m through  choice  of  n.  . 


1 
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j 

i 
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4.2  Formulation  of  the  Inventory  Model  as  a Markov  Inspection  Process 


We  will  now  show  that  the  inventory  process  with  random  production 

described  above,  fits  the  Markov  inspection  model  of  Chapter  2. 

The  underlying  state  of  the  process  is  the  current  inventory  x . 

After  observing  x , we  must  decide  how  many  batches  to  produce.  We 

may  consider  the  decision  as  being  made  sequentially  as  follows: 

Suppose  t batches  have  been  produced,  but  the  number  of  usable  items 
t 

they  contain,  'l  P.  , has  not  yet  been  determined.  This  corresponds 

j=l  ^ 

to  the  observable  state  (x,t)  . We  may  either  produce  an  additional 
batch,  or  terminate  production.  The  former  action  corresponds  to  non- 
inspection, at  cost  C , and  yields  the  new  observable  state  (x,t  + 1)  . 

The  latter  action  corresponds  to  inspection,  since  the  new  state 

t 

(y,0)  , where  y = x-*-  'l  p -C  becomes  known. 

i=l  ^ 

The  transition  matrix  associated  with  production  (non-inspection) 
is  given  by 


The  transition  matrix  associated  with  termination  and  satisfaction  of 
demand  (inspection)  is  given  by 


1‘^x-x’ 

I 


X ^ x’ 

X < x'  . 


The  costs  of  non-inspection  and  inspection,  respectively  are 
0^  “ C and  = h(x)  + k(-x)  . 

With  these  identifications  our  inventory  process  becomes  a 
Markov  Inspection  Process  as  defined  in  2.1. 
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Remark: 


The  model  of  Chapter  2 permits  greater  generality,  in  that  the  size 
of  the  j-th  batch  produced  might  be  allowed  to  depend  on  previous  pro- 

duction  X + I p.  . This  would  give  rise  to  a more  general  transition 
k=l  ^ 

matrix  than  P , given  above.  However,  as  it  is  difficult  to  see  anv 

XX 

application  for  this  added  generality,  we  will  not  pursue  it  further. 

We  seek  to  minimize  the  expected  total  discounted  cost  over  L 
periods,  discounting  at  a rate  of  S per  period.  The  optimality 
equation  (2.2.2)  specializes  to 


(4.2.1) 


i * 

V^(x,t)  = min  |C  + V (x,t  + 1,L); 

I fh(z)  + k(-z)  + ev*(z,0,L  - 1)]J  . 

xy  yz  > 


Having  identified  our  inventory  model  as  a Markov  Inspection  Pro- 
cess, we  wish  to  apply  the  results  of  Chapter  2 to  find  conditions  for 
the  optimality  of  a monotone  policy.  In  other  words,  when  is  the 
optimal  ordering  level  a decreasing  function  of  current  inventory, 
as  intuition  suggests? 

Note  that  the  matrix  P is  upper  triangular.  If  the  distribution 
{pj}  is  such  that  P is  TP2  then  Assumptions  A1  and  A2  of  Chapter  2 
are  satisfied.  (Such  distributions  are  called  Polya  frequency 
functions  of  order  2).  (PF2)  • See  [ 6 ]. 

Furthermore,  lince  the  unit  production  cost  is  c > 0 , the  second 
structural  condition  (SC2)  must  hold.  Otherwise,  for  some  observed 
state  (x,t)  , continual  production  is  optimal.  But  this  policy  has 
infinite  expected  cost,  which  produces  a contradiction. 
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Thus  the  general  structure  theorem  of  Chapter  2 (Theorem  2.3) 
applies,  provided  we  assume  the  first  structural  condition  (SCI). 

Theorem  A. 2: 

For  the  L period  production  problem  with  PF2  production 

distribution  > linear  ordering  cost,  and  general  holding  and 

penalty  costs,  if  there  is  a critical  number  Xq  such  that  ordering 

zero  is  preferred  to  ordering  one  when  x ^ and  not  otherwise, 

(subsequent  orders  being  optimal) , then  the  optimal  order  size 
★ 

n (x,L)  is  nonincreasing  in  initial  inventory  x . 

Furthermore,  the  cost  of  the  policy  ; (order  n , then  proceed 

optimally),  V(x,0,L,tt^)  , is  quasi-convex  in  n . (A  function  f (n) 

* 

is  quasi-convex  if  f(n  + 1)  ^ f (n)  for  n < some  n and 

* 

f (n  + 1)  ^ f (n)  for  n ^ n ) . 

The  fact  that  V(x  ,0,L,ir^)  is  quasi-convex  in  n is  an  immediate 
consequence  of  the  optimality  of  one  step  look-ahead  policies 
(see  (2.3).B 

In  the  remainder  of  this  chapter  we  will  let  V(x,0,L,r  ) = 

n 

* * 

V(x,L,tt^)  and  V (x,0,L)  = V (x,L)  . Also  we  will  deal  only  with 
the  binomial  case  henceforth. 


4.3  The  Binomial,  Single  Period  Case 

We  first  note  that  when  production  is  Binomial  the  associated 
transition  matrix  has  the  form 


XX 


x' 

x' 


X 

X + 1 


i 

I 

I 

i 

\ 

I 


i 


ii 

I' 


1 

f 

y! 

f 

1. 


otherwise . 
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u 


I 


\ 


It  is  immediate  that  P satisfies  hypotheses  Ai  and  A2  of  2.3;  P 
is  upper  triangular  and  totally  positive  of  order  2. 

In  order  to  apply  Theorem  4.2,  it  is  sufficient  to  verify  the 
structural  condition  (SCI).  That  is,  we  must  show  that  there  is  a 
critical  number  such  that  V(x,tt^)  ^V(x,tTq)  for  x < x^^ 

and  V(x,TTj^)  > VCx,-^^)  for  x ^ x^  . (We  suppress  L for  the 
single  period  case) . This  will  imply  that  the  optimal  order  size 

* VC 

n (x)  = 0 for  X ^ and  n (x)  > 0 for  x < . 

To  show  that  the  first  structural  condition  holds,  it  will  be 
necessary  to  make  the  following  assumptions: 


A3  : The  demand  distribution  ^ Polya  frequency 

function  of  order  2 (PF2)  . That  is,  the  matrix 

Q^j  ~ j TP2  (see  Assumption  A2  of  2.3). 

A4 : The  function  il(x)  = — x + h(x)  + k(-x)  is  quasi-convex 
with  a minimum  at  x = 0 , and  strictly  increasing  for 
X ^ 0 . (This  is  equivalent  to  £(x  + 1)  - £ (x) 
satisfying  the  SCP  in  x) . 


Assumption  A3  immediately  yields  the  analog  of  Lemma  2.3; 


Lemma  4.3.1: 


If  ^ ^^^2  distribution  and  m^  satisfies  the  SCP 

in  j , then  ^ j satisfies  the  SCP  in  1 . 


Proof : 


Apply  Lemma  2.3  to  the  matrix  and  make  a change  of 


variables.  ■ 
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Assumption  AA  has  the  following  interpretation:  Consider 

the  certainty  equivalent  problem  where  we  pay  --  per  item  and 
all  orders  are  filled  exactly.  Suppose  demand  in  a period  is 
known.  The  function  J!.(x)  represents  the  relative  cost  of  pro- 
ducing an  excess  x over  demand.  Assumption  A4  implies  that 
i(x)  is  nonincreasing  in  x for  x £ 0 , which  means  that  each 
additional  unit  cost  of  production  — is  more  than  compensated 

for  by  a decrease  in  penalty  (shortage)  cost  k(-x)  . Thus 

c c 

— X + k(-x)  4-  X , X ^ 0 . Clearly,  when  x ^ 0 , — x + h(x)  t x 

(strictly,  for  c > 0) . 

Theorem  4.3: 

For  the  one  period  problem  with  binomial  production.  Assumption 

•k 

A3  and  A4  imply  that  the  optimal  production  level  n (x)  is  a non- 
increasing function  of  initial  inventory  x . Furthermore,  for  each 

X , the  cost  of  the  policy  "produce  n ,"  V(x,ir  ) , is  quasi-convex 

n 

* 

in  n , i.e.  V(x,tt  < V(x,tt  ) for  n < n (x)  and  V(x,r  ) > 

n+1  — n n+i  — 

* 

V(x,TT  ) for  n > n (x)  . 
n — 

Proof : 

By  Theorem  4.2,  we  need  only  show  that  the  first  structural 
condition  (SCI)  is  satisfied,  (SC2)  , (Al),  and  (A2)  having  already 
been  shown.  Equivalently,  we  must  show  that  for  some  , 

V(x,TT^)  £ V(x,iTq)  when  x < x^^  and  not  when  x ^ x^^  . Ke  must 
show  that  u(x)  = V(x,n^)  - V(x,r^)  satisfies  the  SCP  in  x . 

By  Assumption  A4,  for  l(x)  - — x + h(x)  + k(-x)  , the  successive 
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differences  H(x  + 1)  - i(x)  satisfy  the  SCP  in  x . By  Assumption 
A3  and  Lemma  4.3,  so  does 

^ [H(x  + 1 - z)  - H (x  - z)]  = 

E U(x  + 1 - C)  - i(x  - 0>  = 

(4.3.1) 

— + ilj^(x  + 1)  - i^(x)  , where  J.^(x)  = 

E {h(x  - O + k(C  - x)} 

is  the  expected  storage  plus  shortage  cost  when  inventory  is  x 
after  production.  But 


A(x)  = V(x,iT^)  - V(x,tTq)  = 

[c  + qf.^(x)  + pJj^Cx  + 1)]  - ij^(x)  = 
c + p(Hj^(x  + 1)  - H^(x))  . 

Since  the  expression  in  (4.3.1)  is  just  — A(x)  , we  conclude  that 

P 

A(x)  also  satisfies  the  SCP,  and  the  proof  is  concluded.® 

Remark: 

Note  that  no  assumptions  on  the  form  of  h(x)  and  k(x) 
are  made,  other  than  A4 . 

There  is  an  alternate  approach  to  obtaining  the  optimality 
of  a monotone  policy,  assuming  convexity  of  h(x)  and  k(x)  , 
but  without  restriction  on  the  demand  distribution.  This  is  the 

approach  followed  in  Karlin,  Arrow  and  Scarf  [ 4 ].  This  method 

★ 

does  not  extend  to  the  multi-period  case,  however,  because  V (x) 
need  not  be  convex  when  h(x)  and  k(x)  are. 
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Exaoiple  A.  3.1:  (Non-Convexity  of  the  Optimal  Cost  Function  V (x)) 

* * 

Let  V (x)  = V (x,l)  be  the  optimal  cost  function  for  the 

one  period  problem  with  initial  inventory  x . Let  V(x,~  ) be 

n 

the  f.pected  cost  under  the  policy  "produce  n items." 

|3u  u ^ 0 

Suppose  p = — , c = 1 , h(u)  i 0 and  k(u)  =\ 

(0  u ^ 0 

Assume  also  that  the  demand  r r 2 . 

Clearly  V (2)  = 0 and  n^(2)  = 0 . 

For  X = 1 ve  compute  as  follows: 

V(x.Tip)  = k(l)  = 3 

V(x,TT^)  = c + qk(l)  = 2 1/2 

V(x,-rr2)  = 2c  + q^(k(l))  = 2 3/4 

By  quasi-convexity  in  k of  V(x,itj^)  (Theorem  4.3),  we  may 
conclude  that  V (1)  = 2 1/2  and  n (1)  = 1 . 

Similarly,  for  x = 0 we  compute 

V(0,tTq)  = k(2)  = 6 

V(0,T^)  = c + qk(2)  + pk(l)  = 5 1/2 

V(0,rO  = 2c  + q‘k(2)  4 2pqk(l)  = 5 

V(0,tt3)  = 3c  + q\(2)  + 3q‘pk(l)  = 4 7/8 

V(0,TT^)  = 4c  4-  q‘^k/2)  + 4q^pk(l)  = 5 1/8 

A(;ain  by  Theorem  4.3,  jince  V(0,tt^)  < V(0,w^)  and 

* 

V(0,TTj)  V(0,’T.)  , j < 3 , we  can  conclude  that  V (0)  = 4 7/8 

* 

and  n (0)  “ 3 . 


r 


AO 


Thus  V (x)  can  be  tabulated: 


* i 

[ V (x)  : 

n (x) 

1 

0 

A. 875 

1 

3 

1 

2.5 

1 

2 

0 

0 

We  seo  immediately  that  V (x)  is  strictly  concave  on  the  set  {0,1,2} 

When  the  holding  and  shortage  costs  are  linear,  say  h(u}  = hu 

* 

and  k(u)  = ku  , respectively,  we  can  determine  n (x)  by  the  follow- 
ing reasoning:  Suppose  we  have  produced,  but  not  inspected,  n items. 

Let  a(n)  be  the  Binomial  (n;p)  random  variable  denoting  usable 
output.  The  expected  cost  of  producing  an  additional  item  is 
c + hpP(x  + c(n)  ^ , which  is  nondecreasing  in  n . The  expected 

gain  from  producing  an  additional  item  is  kpP(x  + o(n)  < O which 
is  nonincreasing  in  n . It  follows  that 

* 

n (x)  = min  {n  : kpP(x  + o(n)  < ^ c + hpP(x  + o(n)  ^ S) 

= min  |n  : P(o(n)  < C - x)  1 p(i,Vh)  \ * 

From  this  relationship  several  interesting  conclusions  can  be  drawm. 


Lemma  A . 

3.2: 

In 

the  case 

where  all  costs  are 

linear: 

(a) 

n (x) 

depends  on  c , h , 

and 

k 

(b) 

n (x) 

is  nonincreasing  in 

c 

and 

in  k 

c + ph 
p (k  + h) 


(c)  Production  is  optimal  only  when  p ^ p 
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The  dependence  of  n (x)  on  p Is  more  subtle.  The  next 

it 

example  (4.3.2)  shows  that  n (x)  need  not  be  a monotone  function 
of  p for  a given  x , even  when  C = D (deterministic  demand). 

Note  that  it  is  optimal  to  produce  if  and  only  if  P ^ Pq  = * 

it 

provided  x < D . The  optimal  production  level  n appears  to  be 

unlmodal  as  a function  of  p for  fixed  x . Thus,  when  p is  near 

* * 

Pq  , and  when  p is  near  1 , n = D - x , but  n may  be  strictly 

larger  than  D - x for  intermediate  values  of  p . For  p near 

Pq  , it  is  optimal  to  produce  D - x , but  no  "spares."  This  is 

because  p is  so  small  that  the  expected  reduction  in  shortage  cost 

from  producing  a spare  is  less  than  the  unit  production  cost  c . 

* * 

In  Example  4.3.2  the  optimal  cost  V = V (0)  is  a nonincreasing 
function  of  p . This  is  true  in  general  for  the  one  period  case. 

Lemma  4.3.3: 

it 

For  fixed  inventory  x , the  optimal  cost  V (x)  is  a non- 
increasing function  of  p . 

Proof : 

When  the  holding  cost  is  zero,  for  a fixed  ordering  level  n the 
expected  cost  is  cn  + kE  max  (C  - x - a(n),0)  . This  is  a continuous, 
nonincreasing  function  of  p . The  result  follows  immediately  in 

it 

this  case,  since  V (x)  is  the  minimum  of  the  above  functions. 

When  the  holding  cost  h is  non-zero  the  expected  cost  of  an  order 
size  n is  cn  + kE  max  ((;  - x - a(n),0)  + hE  max  (x  + a(n)  - ^,0) 
which  increases  for  p > p and  decreases  for  p < p for  some  p 

•k 

It  can  be  shovm  that  n < n for  p > p^  , which  would  imply  the 


result.  We  omit  the  proof. 
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Example  4.3.2:  Variation  of  Optimal  Order  Sii:e  n (x)  and  Optimal 

A 

Cost  V (x)  as  a Function  of  p for  Fixed  x 

Consider  the  one  period  linear  problem  with  c = 1 , k = 4 , 
h = 0 , X = 0 , and  D = 2 . 

c ★ * 

Note  that  for  p<7“=.25,n  = 0 and  V = 2k  = 8 . For 

— k 

* * 

p = 1 , n =2  and  V = 2c  = 2 . For  intermediate  values  of  p , 

A A A 

n and  V are  given  in  the  following  table.  Note  that  n rises 

A 

to  a maximum  and  then  decreases,  while  V is  continuously  decreasing 
in  the  interval  [0,1]  . 

4.4  The  Multiperiod,  Binomial  Case:  A Counter-Example  and  a Result 

The  result  of  the  previous  section  concerning  the  optimality 
of  monotone  ordering  policies  for  the  single  period  case,  does  not 
extend  without  restriction  to  the  multiperiod  case.  The  following 
example  is  a 2 period  problem  with  linear  ordering,  holding,  and 
shortage  costs  and  a constant  demand,  for  which  a monotone  policy  is 
not  optimal. 

Example  4.4:  An  Example  in  Which  a Monotone  Policy  is  Not  Optimal 

We  set  c = 1 , k(u)  = 3u  , h(u)  = hu  , p = -I-  , and  ^ = 2 . 
Assume  h 1 . 

Consider  first  the  single  period  case.  Note  that  with  h = 0 
this  is  identical  to  Example  4.3.1.  A similar  calculation  shows  the 
following  result: 


I 
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EXAMPLE  4.3.2 

AND  V*  AS  A FUNCTION  OF  P 


* ★ 


p 

1 « 
! n 

K 

V 

CM 

1 

O 

0 

8.0 

.26 

2 

7.92 

.28 

3 

1 

7.73 

.30 

3 

7.51 

.35 

4 

6.97 

.40 

4 1 

6.42 

.45 

4 

5.93 

.50 

1 

3 

5.5 

.55 

3 

5.07 

.60 

3 

4.66 

.65 

3 i 

4.30 

3 

3.97 

.75 

3 

I 

i 3.69 

1 

.80 

3 

3.45 

.85 

1 

2 

1 

3.2 

.90 

2 

2.8 

.95 

2 

2.4 

1.0 


2 


2.0 
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rsi 


cn 


Ui 

pj 


bJ 


C 


A5 


X n (x,l) 


V*(x,l) 


2 0 

1 1 

0 3 


0 

2.5 

4,875  + .125h 


Analysis  of  the  two  period  case  is  dependent  on  the  value 
of  h : 


X n (x,2) 


V*(x,2) 


4 

3 

2 


2h 

2.25  + 1.5h  h 1/2 

2.5  + h h ^ 1/2 

4.46875  + 1. 03125b  h £ .448 

4.875  + .125h  h > .448 


* 

Thus  we  see  that  for  .448  ^h^,5  , n (2,2)  = 0 and 

“k 

n2(3,2)  = 1 . The  optimal  policy  is  not  monotone. 

A further  analysis  of  the  two  period  case  reveals  that  a one- 
stage  look-ahead  policy  is  not  optimal.  Let  be  the  policy: 

order  k during  the  first  period,  then  continue  optimally.  For 
this  example  the  following  behavior  may  be  observed,  given  initial 
inventory  of  2 with  two  periods  remaining. 


”^2 

"2 

^0 

"o 


< 


< 


< 


< 


^0 

^2 


< 


< 


< 


< 


"i 

"i 

"2 


0 £ h j<  .429 

,429  <.  h <_  .448 

,448  < h < .467 


1 


.467  < h 
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Lim:. 


(Here  means  V(2,2,tt^)  < V(2,2,ti^)).  For  .429  £ h _<  .467  , 


V(2,2,tt^)  is  not  quasl-convex  in  n and  the  one  stage  look-ahead 


policy  is  not  optimal. 


Example  4.4b:  A Second  Example  in  Which  a Monotone  Policy  is  Not 

Optimal 


We  modify  the  previous  example  by  setting  the  holding  cost 
h = 0 and  imposing  a discount  factor  6^1. 

We  consider  the  two  period  case. 


n (x,2) 


V*(x,2) 


0 

0 

1 

|0 

I2 


0 

2.56 

1 + 1.256 
4.8756 

2 + 2.468756 


6 £ .8 
6 ^ .8 
6 £ .831 
6 > .831 


Thus  we  see  that  in  the  interval  6 = .8  to  6 = .831  , a 
monotone  policy  is  not  optimal. 


The  above  example,  illustrates  the  effect  of  the  concavity 

•k 


of  V (x,l)  on  optimal  policies  for  the  two  period  case.  The 
closer  initial  inventory  is  to  total  demand  2D  , the  greater  the 
expected  reward  for  ordering  an  additional  item.  This  leads  to 
the  possible  optimality  of  non-monotone  policies  in  he  case  of 
positive  holding  cost  or  discount  factor  6 < 1 . 

However,  in  the  non-discounted  multiperiod  case  with  no  holding 
cost,  a monotone  policy  is  optimal. 
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Before  proving  this  result  we  state  a lemma  which  is  valid 
under  very  general  conditions: 

Lemma  4.4: 

★ 

If  n (x,Jl)  > 0 , then 

V*(x,Jl)  > - + V*(x  + l,i)  . 

- P 

Proof : 

Given  the  option  to  inspect  the  first  item  produced  and  revise 
initial  lot  size  accordingly,  the  optimal  cost  for  i periods, 
starting  with  inventory  x would  be 

* * * 
c + qV  (x,i)  + pV  (x  + 1,11)  (x,H)  , 

The  R.H.S.  represents  the  optimal  cost  in  the  absence  of  such  an 
option. 

Remark: 

This  result  shows  that,  when  it  is  optimal  to  order,  the  value 
of  an  additional  item  in  stock  is  at  least  as  large  as  its 
"certainty  equivalent  price,"  c/p  . 

Theorem  4.4: 

For  the  i period,  undiscounted  problem  with  binomial  production. 


i 


if  there  is  no  holding  cost  (h(u)  H 0)  , then  assumptions  A3  and 
A4  imply: 
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* 

(a)  The  optimal  initial  production  lot  size  n (x,i)  is 
a nonincreasing  function  of  initial  inventory  x . 

(b)  The  expected  cost  of  the  policy  it  : produce  n , 

n 

then  choose  all  subsequent  lot  sizes  optimally, 

V(x,il,iT^)  , is  quasi-convex  in  n . 

Proof : 

By  Theorem  4.2,  we  need  only  show  that  A(x,l)  satisfies 

the  SCP  in  x . Equivalently  we  must  show  the  existence  of  a critical 

number  x^^  such  that  V(x,J.,tt^)  ^ V (x , J!,  , tTq)  for  x < x^^  and 

V(x,8,,7t^)  > V(x,Z,7Tq)  for  x ^ . 

Proceeding  inductively  on  S.  , v;e  note  that  the  result  has 

previously  been  shown  for  i = 1 . Assume  that  A(x,H  - 1)  satisfies 

the  SCP  in  x and  {x  : A(x,2.  - 1)  > 0}  ^ 0 . 

* 

By  the  induction  hypothesis,  n (x,il  - 1)  = 0 for  x ^ x^^  . 

* 

Thus  V (x,Z  - 1)  is  also  the  expected  J?.  - 1 period  cost  when 
inventory  is  x after  initial  production.  For  x ^ x^^  ^ , 

V(x,l  - 1,^q)  < V(x,J,  - l,r^)  , or  equivalently 

V (x,i-l)  <c+pV  (x+l,£-l)+qV  (x,£-l)  . 


49 


By  Lemma  4.4  and  monotonicity  of  k(-.x)  , m(x)  is  nonincreasing 
for  X ^ . We  have  shown  that  m(x  + 1)  - m(x)  satisfies 

the  SCP  in  X . As  in  the  proof  of  Theorem  4.3,  this  implies  that 
A(x,Jl)  = pE  {m(x  + 1 - - m(x  - C)}  satisfies  the  SCP  in  x . 

To  complete  the  induction,  it  will  suffice  to  prove  that 
Xj  = min  {x  : A(x,!,)  >0}  ^ ^ . We  have  shown  that 

m(x  + 1)  - m(x)  ^ 0 for  x ^ x^  ^ . Thus  A(x,i)  = 
pE  {m(x  +1-0  - ni(x  - E)}  ^0  for  x ^ x^^^  . It  follows  that 

x^  ^ x^  ^ , which  completes  the  proof.  ■ 

We  have  also  proven  the  following: 

Lemma  4.4.2: 

Under  the  hypotheses  of  Theorem  4.4,  if  it  is  optimal  to  order 
when  stock  is  x and  i periods  remain,  it  is  also  optimal  to 
order  when  i'  period  remain,  !,'  > 1 . 

Remark: 

We  have  not  been  able  to  show  that  the  optimal  production  level 

* 

n^(x)  is  nondecreasing  in  2,  . This  seems  to  be  an  open  question. 
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CHAPTER  5 

SUMMARY  AND  CONCLUSIONS 

We  present  a general  model  of  a process  which  requires  costly 
Inspection  in  order  to  determine  the  state.  At  any  time  two  alterna- 
tives are  available:  to  inspect  the  process  (which  may  alter  it), 

or  not  to  inspect. 

The  optimal  time  interval  between  successive  inspections  is 
shown  to  be  a decreasing  function  of  the  state  at  last  inspection, 
under  conditions  specified  herein. 

Our  model  is  applicable  to  a wide  variety  of  deteriorating 
processes,  including  repairable  and  non-repairable  machines,  and  a 
class  of  inventory  problems  with  uncertain  ordering. 

The  results  presented  illuminate  the  structure  of  optimal 
Inspection  policies  and  are  useful  in  computing  such  policies. 


I 
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